Instabilities of quadratic band crossing points 
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Using a functional renormalization group approach, we study interaction-driven instabilities in 
quadratic band crossing point two-orbital models in two dimensions, extending a previous study of 
Sun et al. [Phys. Rev. Lett. 103, 046811]. The wavevector-dependence of the Bloch eigenvectors 
of the free Hamiltonian causes interesting instabilities toward spin nematic, quantum anomalous 
Hall and quantum spin Hall states. In contrast with other known examples of interaction-driven 
topological insulators, in the system studied here, the quantum spin Hall state occurs at arbitrarily 
small interaction strength and for rather simple intra- and inter-orbital repulsions. 
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I. INTRODUCTION 

Topological insulators are a new state of electron mat- 
ter that have attracted enormous interest recently '1']. 
Similar to the integer quantum Hall effect, these states 
can be distinguished from conventional insulators by 
topological invariants and robust sub-gap surface states. 
Topological band insulators cannot be continuously de- 
formed into a topologically trivial state without closing 
the band gap. 

In two spatial dimensions and in the presence of time- 
reversal symmetry, topological insulators are character- 
ized as quantum spin Hall (QSH) states If the com- 
ponent of the electron spin along a certain quantization 
axis is conserved, the picture of the QSH state is espe- 
cially simple. Then, the currents in the edge states cause 
opposite Hall conductivities for spin up and spin down, 
respectively, which results in quantized spin-Hall trans- 
port, with the conductance determined by a topological 
quantum number per spin obtained from the bulk band 
structure. This simple QSH state can be viewed as two 
copies of Haldane's 1988 quantum Hall state without net 
magnetic field 0, with the two copies being related to 
each other by time reversal. In the case of broken time- 
reversal symmetry, e.g., if one leaves out one copy or re- 
verts its orbital part, one obtains a related state, known 
as quantum anomalous Hall (QAH) state. This state 
exhibits a quantized charge Hall conductance at zero ex- 
ternal magnetic field (see, e.g. Rcf. [3]), as in Haldane's 
initial proposal for the spinless model. In both, QSH 
and QAH, the occurrence of edge states through which 
the electrons propagate comes as a direct consequence of 
the topologically nontrivial nature of the bulk electronic 
band structure JJ. 

Usually, this topological structure of the band struc- 
ture state is caused by the presence of spin orbit cou- 
pling. Initially, Kane and Mele proposed a spin-orbit 
term on the graphene lattice Q, but it turned out that 



for graphene, the effect would be too weak [1, Q. Based 
on an insightful theoretical proposal 0, the QSH state 
could instead be realized in HgTe/CdTe quantum wells, 
where hallmarks of the QSH effect could be experimen- 
tally observed the first time [1, 8]. 

All this beautiful physics can be understood in the 
single-particle picture. One could ask, however, if simi- 
lar topologically non-trivial states can arise from interac- 
tions, through a phase transition at some critical temper- 
ature. Following this idea, different simple models were 
analyzed. Raghu et al. Q considered the honeycomb 
lattice with strong next-nearest neighbor repulsion using 
mean-field and functional renormalization group (fRG) 
techniques. Above a nonzero critical interaction strength 
, the quantum-spin Hall state was found as the ground 
state of the system. This instability implies a sponta- 
neous breaking of the spin rotational invariance. While 
this study serves as a proof of principle, it is by no means 
clear how one can realize the rather peculiar interaction 
with strong second nearest neighbor repulsion. Another 
study was performed by Zhang et al. [To| for a three- 
dimensional lattice model where third-nearest neighbor 
interactions were essential to stabilize the QSH state. A 
promising proposal came from Sun et al. (llj who ar- 
gued, based on mean-field calculations, that quadratic 
band crossing point (QBCP) models should host QSH 
states already at arbitrarily weak interactions. The dif- 
ference to the honeycomb lattice with its Dirac points 
is that, for a QBCP, the density of states is nonzero at 
the crossing point, and, hence already a small interac- 
tion suffices to drive an instability. This picture is also 
supported by a mean-field study of Wen et al. who 
found topologically non-trivial phases already for small 
interactions in Kagome and decorated honeycomb lattice 
models with quadratic band crossing points. 
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In this work we consider certain QBCP models [11 1 
in two dimensions using fRG methods. This allows us to 
explore the possibilities for interaction-driven topological 
states beyond mean-field theory. We furthermore inves- 
tigate if there can be superconducting states emerging 
from topologically non-trivial insulators. 

We briefly note that these interaction-driven topologi- 
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cal insulators are also referred to as topological Mott in- 
sulators. The same term is sometimes used in the study 
of a slightly different question, which is the interplay be- 
tween a Kane-Mele spin-orbit term leading to a topolog- 
ical band insulator and the Hubbard onsite interaction 
leading to a Mott insulator, if sufRciently strong [T3l.[l3|. 
Our paper addresses the interaction-induced generation 
of a Kane-Mele-type term in absence of significant spin- 
orbit coupling in the bare Hamiltonian, by interactions 
that are weaker than those required to drive the sys- 
tem Mott-insulating. A recent work [l^ addresses the 
transition between topologically non-trivial and trivial 
states when nonzero interaction parameters are changed 
in finite-size systems. Interestingly, the single-particle 
gap can remain robust across the topological transition, 
and both sides of the transition are insulators. 



II. CONTINUUM FERMION MODEL 

We first consider a spin-1/2 model in the continuum 
in two dimensions, which describes the neighborhood of 
the QBCP in a quite general way. Following Ref. |11J let 
the Hamiltonian be given by 

H ~ -fffrcc + -ffint, (1) 

where the free part reads as 

i/free = E / , (k)^O^, (k)^^, _^ (k) 

0,0' ,s 

= j (ik*^(k)H°(k)*(k). (2) 

Here the index s denotes spin and o = 1,2 denotes 
two different Fermi fields, which should be identified 



with the orbital degree of freedom. In the second 
line we have written the Hamiltonian in matrix nota- 
tion so that * = {'4>AA,'4>B.:\,'4'A,i,'4^B,\)'^ and ft = 
^, V'b t' 4,' combines spin and orbital degrees 
of freedom. We choose the free part to be of the form 

(k) =Is® [di (k)/ + (k)t7, + 4 (k)a,] , (3) 

where Is is the unity matrix in spin space, I the unity 
matrix in orbital space, and Ux and cr^ the Pauli-matrices. 
The integral is over a disk in two-dimensional momentum 
space. The disk radius, i.e. the ultraviolet (UV) cutoff, 
just determines the energy window focused on and will 
not be of any qualitative importance. The coefficients are 
d/(k) = ti{kl + kl) - fj., d^(k) = 2txkxky and d^(k) = 
tz{kl — ky). Usually, we will set t/ to zero and tx = tz = t 
for simplicity. There is no ty as this would break time- 
reversal symmetry or already create a QSH state at the 
bare level. Except for Sec. I VII the chemical potential is 
set to zero so that the QBCP lies at the Fermi level. Note 
that we exclude the possibility of spin orbit coupling by 
restricting the Hamiltonian to unity in spin space. 

The free part can be easily written in a diagonal ba- 
sis. With a proper transformation of the Fermi fields 
7„,s(k) = J2o^n.oi'^)'4'o,sO<-), where ^n,s is the trans- 
formed field in band n with spin s, the free Hamiltonian 
becomes 

H°(k) =/s® [d/(k)/ + <(k)a,], (4) 

with d[ = \^dx + d^. The band structure of the model 
consists of two parabolas, which have a QBCP at the ori- 
gin (see Fig. [1]). In our choice tx = tz = t, the dispersion 
is rotationally invariant in the plane. 

The interacting part of the Hamiltonian contains local 
intra- and interorbital repulsions. 



+ Y E /rfkdk'dqV^t,(k)V,t,^^,(k')7^o^,.(k'-q)V'o,.(k + q) 

(ki,k2,k3,k4)V't^,,(k3)<,,,(k4)Vo„s'(k2)^Oi,.(kl). (5) 

oi ,02,03 ,04 

I 



If we use the basis in which the free Hamiltonian is di- 
agonal, we have to transform the interacting part of the 
Hamiltonian accordingly: 

Kiin2ii3n4 (kl, k2, ka, k4) 

= E V;i020304(ki,k2,k3,k4) 

01502,03,04 



X M„,oi(ki)w„,o,(k2)<303(k3)<^„^(k4) (6) 

so that the local interaction ([S]) assumes a pronounced 
momentum dependence in the new basis. The pref actor 
consisting of the four u„o(k)s is sometimes referred to as 
orbital make-up. 

As pointed out in Ref. 11] , the QBCP carries a vortex- 
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like winding of the Bloch eigenvectors (with two compo- 
nents for the amplitudes in the two orbitals) that cannot 
be made undone continuously. The core of the vortex 
needs to be a degeneracy point, and hence the QBCP 
cannot be removed easily. When we think about the sta- 
bility of the QBCP with respect to interactions, there are 
essentially two ways to remove the QBCP which we will 
find to be realized depending on the interaction param- 
eters. The free Hamiltonian has rotational invariance in 
the kx , fcj,-plane. One way is now to break this rotational 
symmetry by splitting the QBCP into two Dirac points. 
In terms of the eigenvectors this stretches out the wind- 
ing along a branch cut connecting the two Dirac points 
with an inversion of the direction of the Bloch eigenvec- 
tors. The other way is to open a gap by breaking either 
time reversal symmetry in a given spin sector. This cor- 
responds to a nonzero dy-term. Only when the dyS in 
spin-up and spin-down sectors are opposite in sign but 
of the same magnitude, time-reversal symmetry is still 
present. This state is then a QSH state. Otherwise, time- 
reversal symmetry is broken. If dy is spin- independent, 
we have a QAH state. 

Next we study the instabilities of a continuum QBCP 
model and of a related lattice variant in the spin-1/2 case 
by a fRG approach. This extends the mean-field study 
of Sun et al. [TT| of a spinless model. We find that 
a single QBCP might be the most favorable situation 
to realize spontaneous QSH instabilities. Compared to 
the previous studies neither particular longer-ranged nor 
rather strong interactions are needed, and no threshold 
value for the interaction strength needs to be exceeded. 

III. FUNCTIONAL RENORMALIZATION 
GROUP TREATMENT 

In this section we describe the momentum cutoff fRG 
method [iGl - flsl l. which is employed in this work. We 
consider the action with fermionic Grassmann fields 7^ (p) 
and 7s (p), 

+ [ dpidp2dp3^V{pi,p2,P3,n4) 

s,s' ri4 
Xls{P3)ls'{PAhs'{P2hs{Pl), (7) 

where we used the combined index p — (k; w; n) with the 
Matsubara frequency uj, the band index n as above and 
/ dp = J dk^^ s = ±i is the spin projection. V, 
is the Brillouin zone volume, or the considered volume in 
momentum space. In the second term, the interaction, 
we have used wavevector-frequency conservation, and it 
is only summed over the fourth band index 714. 

The idea of the fRG method is to let the quadratic part 
of the action depend on a cutoff parameter, so that 

Q^(p) = [C^(k, n)] [-li, + e(k, n)] (8) 



with the dispersion e(k, n = 1,2) = d/(k) ± d'j,(k) ac- 
cording to Eq. (HI). For the cutoff- function C^(k, n) we 
choose 

C^(k,n) = e(|e(k,n)| - A). (9) 

O is a step function, which is slightly softened in the 
numerical implementation. With this choice the modes 
with energy below A are cut off, so that the free prop- 
agator Gq{p) = — is restricted to the high 
energy degrees of freedom. At A = Ag larger than the 
band width all perturbative corrections are set to zero, 
as all internal lines in the corresponding diagrams van- 
ish. Hence the correlations and vertex functions of the 
theory at Aq are precisely known. The fRG flow from 
A = Aq down to lower A provides the change of these 
functions, when the perturbative corrections are succes- 
sively added to the system. At A = the full action 
would be recovered, but in the cases and approximations 
below, the flow of some correlation or vertex functions, in 
part corresponding to effective interactions, diverge be- 
fore at a critical scale Ac. This flow to strong coupling 
is indicative for a change of the ground state. The ba- 
sic aim of the fRG flows used here is to flnd out which 
classes of vertex functions or effective interactions drive 
this flow to strong coupling. From this we obtain tenta- 
tive ground state diagrams, as described in the sections 
on the results. We note that the flows to strong cou- 
pling are to large extent physically meaningful artefacts 
of the approximations employed here. If the flow of the 
fermionic self-energy was included, a gap would open at 
the instability scale. This would regularize the diver- 
gence at nonzero scale. Unfortunately, such flows into 
the symmetry-broken or massive sector are by far more 
complicated when the competition of various instabili- 
ties or fermionic gaps is kept (which is our goal here). 
It gets more feasible when only one type of fermion gap 
is considered [2^. A recent work by Sinner and Ziegler 
(25! shows how the functional RG in partially bosonized 
form (this formalism is e.g. reviewed in Ref. [13) can 
be used to determine the final interacting gaps in mono- 
and bilayer graphene with Coulomb interactions, when 
all modes are integrated out. 

The fRG flow for the vertex functions can be obtained 
from an exact flow equation for the generating func- 
tional of the one-particle irreducible correlation functions 
[IE [13 [3 • This generating functional can be expanded 
in the fields, thus yielding an infinite hierarchy of integro- 
differential equations for the vertex functions. In order 
to apply the fRG method to our problem, we have to 
truncate the infinite system by setting the six-point ver- 
tex to zero. We also neglect the flow of the self-energy 
which will only become large, when the interactions flow 
to strong coupling. We then arrive at the fRG equations 
for the coupling function V^{pi,p2,P3,n4), which in a 
spin-rotationally symmetric situation completely deter- 
mines the interaction vertex. This equation reads as 

-jrV^ipi,P2,P3,n4) =rpp+T^H,d + '^PH,cr^ (10) 
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with the particle-particle channel 

n' 

X L^{p,qpp)V^{p,qpp,P3,ni) , (11) 
the direct particle-hole channel 

TpH4{Pi^P2,P:i,ni) ^ -T j dp^\^-2V^{pi,p,pz,n') 

X L'^(p, qpHA)V^{qpH,d,P2,P,ri4) -\- V^{p,pi,p3,n') 
X L^{P, qPHA)V^{qpH,d,P2,P,n^) + V^{pi,p,p3,n') 
X L^{p,qPHA)V^{p2,qpH,d,P,n4) , (12) 

and the crossed particle-hole channel 

Tph,ct{pi,P-2,P3.^'>T'4) ^ ~T j dp^V^{p,p2,P3,n') 

n' 

X L^(j},qpH,cr)V^{pi,qpH,cr,P,n4:) , (13) 

where qpp = (-k + ki + k2; -w + wi + W2; n'), qphM = 
(k-l-ki-ka; w-|-wi-W3;n'), gp/i^cr = (k+k2-k3; w+W2- 
W3] n') are the quantum numbers of the second loop line, 
p = (k; w; n) are those of the first line. Note that we also 
have to sum over the band index n' of the second internal 
line, whereas the momentum and frequency are fixed by 
conservation. The fourth momentum and frequency in 
the interaction vertex is fixed by conservation, so that 
here only the band index is written. The internal loop is 
given by 

L^{p,p') = 5^(p)G^(p') + G^(p)5^(p') , (14) 
with the single-scale propagator 

d 



S^'{p)^G^\p)^ — Q'\p)jG^\p), (15) 

which has support only at the scale A, as the A-derivative 
of the cutoff-function is only nonzero there. In our ap- 
proximation of neglecting self-energy corrections, the full 
propagator G^{p) is identical to the free propagator. 

The frequency dependence of the vertex is ignored in 
this work, as we are mainly interested in low-frequency 
vertices that are relevant for static ordering. Thus the 
frequency summation in the fRG equation can be per- 
formed easily analytically. The integration over momen- 
tum space is done numerically. For this purpose we divide 
the momentum space in several patches and compute the 
coupling function only on a finite set of momenta on 
a ring around the QBCP. in analogy with previous A^- 
patch fRG studies [ig, [2^ . Thereby, the coupling func- 
tion is set constant within each patch (see Fig. [T]). The 
advantage of setting the set of momenta on which the 
coupling function is computed on a ring is, that we can 
take into account the effect of orbital make up, i.e., the 
variation and winding of the Bloch functions around the 





FIG. 1: Left: Dispersion relation of the continuum model 
with the QBCP. Right: Sketch of the patching-scheme: the 
central red dot is the QBCP, and the black dots denote the 
momentum vectors k associated with respective patch, for 
which the coupling function is computed. 



QBCP. The Bloch functions enter the coupling function 
in band language via the orbital makeup in Eq. ([6|). 
This variation can be important for the angular varia- 
tion of gap structures |2l| and can certainly change the 
competition between different instabilities. We note that 
other RG studies [2^ of related problems with two 
QBCPs contract the wave- vector dependence directly to 
the Fermi point, i.e. form one big patch and parametrize 
the interaction by a handful of coupling constants gs for 
the scattering between the Fermi points. While we do 
not have evidence for a failure of this procedure for the 
QBCP model, in the iron pnictides, a similar approxi- 
mation overlooks the angular gap structure [2l|, and, in 
principle, the contraction of a winding Bloch function on 
the center of the winding is ill-defined, at least for a sub- 
set of interaction processes. Moreover, our more refined 
patching schemes allows us to study the doped case as 
well, where the Fermi surface is a small ring around the 
QBCP point. 

We have checked that additional radial patches (i.e., 
introducing a second ring) do not change the qualitative 
results stated in this work. Note that in the continuum 
the UV cutoff, i.e., the starting scale Aq, is somewhat 
arbitrary. Therefore, we choose some fixed UV cutoff. 
The explicit calculations confirm the expectation that 
this choice only affects the absolute value of the critical 
scale, but not the type of instability. 



IV. EMERGENT ORDER PARAMETERS AND 
PHASE DIAGRAM IN THE CONTINUUM 
MODEL 



By investigating the fRG fiow of the effective interac- 
tion for different ratios of U and U' for the QBCP at the 
Fermi level we usually encounter fiows to strong coupling 
at some critical scale Ac. The main growth is in particle- 
hole channels with zero momentum transfer, leading to 
very sharp structures in the effective interactions (see 
Fig. 131). If we focus on this dominant zero- momentum 
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transfer component and drop the remaining interaction 
terms, the effective interaction becomes infinite range in 
real space and decomposes into a product of two fermion 
bilinears, each with one creation and one annihilation 
operator at the same wave-vector. This interaction can 
then be solved in mean-field theory. Then, the bilinears 
correspond to order parameters of the strongly coupled 
phase, the emergence at low scales of which is indicated 
by the flow to strong coupling. In more detail, we find, 
depending on the interaction parameters, the following 
order parameters: 

QsN = yrfk(vl/t(k)(f®a,)vi/(k)) (16) 

s,s' 

*QAH = ydk(*t(k)(/®a,)*(k)) (17) 
= / dk^(z[^l,^,(k)V'A,.(k)-V'LW^B,.(k)]), 

^QSH = ydk(*t(k)(f ®a^)*(k)) (18) 

= JdkJ2{fs.s' I [^L(k)V'A.'(k) -<,(k)Vs,.'(k)]), 

where the brackets (. . .) denote thermal averages. 

Here r are the Pauli matrices in spin space. If we 
choose T in the ^-direction, the state with non- vanishing 
QsN is characterized by a mean-field that is odd both in 
the two orbitals and in the spin projection along the z 
axis. For a given spin component, the QBCP splits into 
two Dirac points, either along the axis or along the 
ky axis. For the opposite spin projection, the dispersion 
is rotated by 90°. The translational symmetry however 
remains conserved. Therefore this state exhibits spin ne- 
matic (SN) order. 

The order parameter $qah describes a QAH phase, 
in which time reversal symmetry is broken. This state 
has a gapped bulk spectrum, as can be found out from 
adding the mean-field to the free Hamiltonian. The state 
then has a quantized Hall conductivity and topologically 
protected edge states, as can be understood, e.g. by com- 
puting the Hall conductivity from the skyrmion-number 
formula and by looking at finite systems in real space Q . 

The QSH phase corresponds to the third order param- 
eter ^QSH- This phase is also gapped and breaks spin 
rotational symmetry. It has helical edge states, which 
results in quantized spin Hall conductivity. 

Now let us discuss the parameter regions where these 
orders emerge in the continuum model in more detail. 
When we run the fRG with the local interaction param- 
eterized by U and U' as initial conditions, we find differ- 
ent effective interactions dominated by the three types 



of ordering listed above in Eqs. p6|) . p7|) and p8|) as a 
function of U' /U . From analyzing which component of 
these three grows most strongly we can deduce a tenta- 
tive phase diagram describing the leading ordering ten- 
dencies. Here, the critical scale Ac serves as an (upper) 
estimate for possible ordering temperatures, or at least 
for the onset of strong correlations of the type indicated. 
Note that the question as to whether these orderings ac- 
tually occur in true long-range form or are prohibited by, 
e.g., collective fluctuations is not answered in this fRG 
scheme. Nevertheless, the analysis is expected to give a 
realistic account of the dominant non-local correlations. 
Another interesting possibility would be that the single- 
particle Hamiltonian already contains small terms corre- 
sponding to the addressed mean fields, e.g., due to spin- 
orbit coupling, and that these terms then get strongly 
enhanced at low temperatures due to the interaction ef- 
fects monitored by the fRG. 

The phase diagram obtained for the QBCP model is 
shown in Fig. [5J For small values of the interorbital 
repulsion U' we encounter the SN phase with suggested 
order parameter (|16p . For t/' > ?7/2 the QAH-order pa- 
rameter (|17p is leading and above U' > 0.7U the system 
is unstable toward the QSH-order ([T5| . 

In Fig. Owe show a typical plot of the effective inter- 
action at the critical scale in the QAH phase. One nicely 
observes the sharp momentum dependence of the q = 0- 
instability in the strong horizontal features. Note also the 
different sign of the couplings with oi = 02 and oi ^ 02 
respectively. This reflects the fact that the expectation 
value of ipl^ ^(k)V'_B.s(k) is imaginary and thus shows that 
the corresponding QAH mean-field breaks time reversal 
symmetry. The less pronounced vertical features corre- 
spond to a tendency toward the emergence of the SN 
phase, which is still present but weaker than the lead- 
ing QAH instability. The other instabilities can also be 
inferred from analyzing these snapshots of the fRG flow 
near the instability for different values of the interaction 
parameters. 

Upon changing U' /U, Ac does not decrease signifi- 
cantly between different phases. This suggests that there 
is no direct competition between the different tendencies, 
and the phase transitions are first order. 

In order to answer the question as to whether there is a 
critical interaction strength needed for the emergence of 
long range order, we also investigated the critical scale in 
dependence of the bare coupling strength for fixed U' /U 
in all three regimes. We observed that the couplings do 
indeed diverge down to an interaction strength of less 
then U — 2t for a bandwidth of 20t. Although an arbi- 
trary small interaction strength is not numerically acces- 
sible, we expect the instability to persist to infinitesimally 
small interaction. 

The conclusion we can draw from this study is that 
this QBCP model is a favorable situation for the occur- 
rence of spontaneous spin Hall instabilities. No particu- 
larly strong or exotic interaction is needed, and the ratio 
U' /U < 1 between interorbital and intraorbital interac- 
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FIG. 2: Tentative £RG phase diagram of the continuum model 
as a function of the interorbital interaction parameter U' in 
units of the intraorbital repulsion. Ac denotes the critical 
scale, defined as the scale at which the couplings exceeds three 
times the bandwidth, obtained with a £RG calculation using 
= 48 patches per band, upper cutoff Auv = lOt and U = 

8t 



tions should not be too unrealistic. The main difficulty 
regarding possible realizations of these phases is the dis- 
persion with the QBCP. We will come back to this in the 
conclusions. 

Note that the d-wave-like wave-vector dependence of 
the components of the free Hamiltonian and its off- 
diagonal content that lead to a wave-vector- variation of 
the Bloch eigenvectors are essential for obtaining these 
interesting instabilities. If we had just taken a Hamilton 
matrix H°(k) oc k^CTz that leads to the same dispersion 
but constant Bloch eigenvectors, the dominant instabil- 
ities would have been Stoner ferromagnetism at small 
U'/U and orbital ordering at larger U'/U. These states 
are neither SN nor support topologically protected edge 
states. 








FIG. 3: Effective interaction near the critical scale in the 
QAH phase. The angle (j>\ ((^2) of the first (second) ingoing 
leg is plotted on the y (x) axis, with the discretization shown 
in Fig. [1] (j)z = Ti IS kept fixed. In our convention the first and 
the third line have the same spin. The upper plot shows the 
vertices with the orbitals oi = 02 , 03 = 04 7^ oi , the lower plot 
shows the combination oi = 04, 02 = 03 7^ oi. The somewhat 
weaker vertical features at (j)2 = (^3 belong to remnants of the 
SN instability that is predominant at smaller U' . 



V. QBCP ON A CHECKERBOARD LATTICE 

For possible realizations of the unconventional particle- 
hole instabilities found in the preceding section, it would 
further be advantageous to have a lattice model that 
shows similar physics. In this section we want to ana- 
lyze a QBCP on a checkerboard lattice, which has two 
sublattices A and B playing the role of the orbital degree 
of freedom in the last section. This model was already 
studied b y S un et al. in mean-field theory for spinless 
fermions Here, we treat the spinful case with the 

fRG. The free Hamiltonian is still of the form of Eqs. ^ 



and ([3]) but with the new hopping terms 

di = 2t/(cos kx + cos ky) — /z , 
k k 

dx = 8ta; cos "f" ' 

dz — 2tz(coskx — cos ky) . (19) 

Here, the lattice constant has been set to unity, tx corre- 
sponds to hopping between nearest neighbors and t/ and 
tz correspond to hopping between next nearest neighbors, 
that is nearest neighbors on the same sublattice. If we 
again set tj — 0, then the hoppings between next nearest 
neighbors connected by a line in Fig. |3]and not connected 
by a line have opposite sign. We also set tx = — t and 
/i = as in the last section. 



This lattice model has a QBCP around the corner of 
the Brillouin zone (tt, tt) and in its vicinity the lattice 
model assumes the form of the continuum model in the 
last section. The dispersion is shown in the right panel 
of Fig. m Further away from the band crossing, the 
dispersion breaks rotational symmetry around (tTjTt). 

We consider an onsite repulsion U and a spin- 
independent nearest-neighbor repulsion, thus, the inter- 
action part of the Hamiltonian reads as 



= ^ E <.(k)^I,,,(k')V'o,.'(k' - q)Vo,s(kH-q) 



k,k',ei 



AN ^ 



COS ■ 



-|- COS 



^ \ 2 2 

k.k'.q ^ 

X 7^t,(k)^t^_^,(k')^^,_^,(k'_q)^„^(k + q), (20) 

where we included an additional factor 1 /4 in the second 
term to compensate that each lattice site has 4 neighbor- 
ing sites, so that the results are directly comparable to 
the continuum model. 

We carry out a similar fRG-analysis as described in 
Sec. ini Again, for small U' jU we encounter a SN phase 
which has the same order parameter as in the continuum 
model given by Eq. (fT6|) . The mean fields of the QAH 
and QSH phases suggested by the fRG, however, have 
an additional prefactor due to the local separation of the 
orbitals and can be written as 



$QAH = E ® ^y)"^^^) ( - 2 Sin ^ sin ^ 



<xxx> 

■OOOO' 
■OOOO' 




FIG. 4: Left: Sketch of the Checkerboard lattice with two 
sublattices A and B indicated by the two different colors. The 
arrows indicate the current pattern in the QAH and QSH 
phase. In the QSH phase the current for opposite spins is 
reversed. Right: Dispersion of the lattice Hamiltonian of Eq. 

dim. 



(21) 
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0.4 0.6 
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^QSH = ^ (^^■(k)(f «) CTj,)^'(k) ( - 2 sin y sh 

k 



j,S,s,s' 



(22) 



with j denoting the lattice sites of sublattice A, 5 — 
— 1 and Z?±(x _ H^, — —1. Note that the 



-\-X _\_v D , 



topological nontrivial phases do not violate local charge 
conservation despite their unusual appearance. It can be 
shown that the expectation value of the fermion number 
operator at a given coordinate remains constant. The 
current pattern in the QAH state is indicated by the 
arrows shown in Fig. S) In the QSH state we get the same 
pattern for one spin component, but here the current for 
the other spin component is reversed. 

The phase diagram of the lattice model is shown in 
Fig. [SI It is similar to the one of the continuum model. 



FIG. 5: Phase diagram of the lattice model as a function 
of the interorbital interaction parameter U' , in units of the 
intraorbital repulsion U = 6t. Ac denotes the critical scale, 
defined as the scale at which the couplings exceeds four times 
the bandwidth, obtained with a fRG calculation using A' = 48 
patches per band. 



both phase transitions occur at similar [/'/[/-ratio. As 
expected the difference between the two models, which 
lies only in the high energy modes, do not play an essen- 
tial role for the determination of the leading instability. 
The low energy instability of a model with a QBCP at 
the Fermi level can be reasonably well approximated by 
the continuum model and is presumably quite indepen- 
dent of details of the band structure in the high energy 
sector. 
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VI. FLOWS AT NON- VANISHING CHEMICAL 
POTENTIAL 



We now want to investigate the instabilities of the 
QBCP at a finite value of the chemical potential so that 
the QBCP is shifted away from the Fermi surface. We 
mainly consider the continuum model of Sec. |IT1 However 
we checked that the results remain qualitatively similar 
in the checkerboard lattice of the last section, though. 

The natural choice for the patching-points of the dis- 
cretization scheme is to set them on the circular Fermi 
surface that opens upon changing the chemical potential 
away from zero, as we are interested in the effective model 
at low energy. We choose the patching points of the other 
band without Fermi surface to be at the same positions 
in momentum space. Again we also used a momentum 
discretization with additional radial patches. 

In all three regimes, we find a critical chemical poten- 
tial /icrit, above which the leading instability is supercon- 
ducting. This means that the leading divergence is now 
for momentum combinations that have total incoming 
momentum, ki + k2 =0 (assuming that the QBCP is at 
the origin in momentum space) . Typical phase diagrams 
are shown in Fig. [51 We see that the critical scale drops 
for larger /i. 

A further interesting result is, that in the regime, 
where we get the QAH for = 0, there is always an 
intermediate phase, either SN or QSH type, before the 
superconducting channel becomes strongest. The lead- 
ing instability of the intermediate phase is determined 
by the ratio of U' and U: above U'/U = 0.6, it is QSH 
and below it is SN. 

We also addressed the question as to which type of 
pairing symmetry can be associated to the superconduct- 
ing instability. It turns out that this question can not 
be conclusively answered within our approach. Depend- 
ing on numerical details, in particular, the discretization 
scheme (i.e., use of one or more radial patches), we obtain 
two different results, which are most plausible according 
to our data and which both have an orbital dependence 
of the order parameter. In the first case, the pairing has 
an interorbital s-wave-like component and a d-wave-like 
intraorbital component. The second, nearly degenerate, 
possibility is an odd-parity, i.e., triplet pairing, symme- 
try that is p-wave-like in all orbital combinations, which 
has relative phase shifts between the orbital components. 
From our data, it seems that different superconducting 
channels are competing, so that numerical details decide 
which one is leading. As the model studied here has so 
far not been realized experimentally, we refrain from a 
classification of the pairings in this two-orbital situation 
and from elaborating this situation with refined numerics 
further. However, a general conclusion from this study 
is that, upon doping, the interaction-driven instabilities 
in such QBCP systems generically give way to unconven- 
tional superconducting phases. 
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FIG. 6: Phase diagram of the continuum model as a function 
of the chemical potential /i at U'/U = 0.55 (upper plot) and 
U'/U = 0.75 (lower plot) 



VII. CONCLUSION 

In summary, we have investigated the weak coupling 
instabilities of the QBCP in a two-dimensional fermionic 
spin- 1/2 system with an angle- resolved fRG calculation. 
Comparing the leading instability for different ratios of 
intraorbital and interorbital interactions, we obtained 
similar results for the continuum model and the model 
on a checkerboard lattice. For weak interorbital inter- 
action, the system is unstable toward a spin-nematic 
(SN) phase. For intermediate U', we encounter the time- 
reversal-symmetry breaking QAH phase, and for a strong 
interorbital repulsion, the leading tendency is toward 
the topological nontrivial QSH phase. Upon moving the 
chemical potential away from the band crossing point, the 
exotic instabilities are replaced by unconventional pairing 
instabilities at lower critical scale. 

Our results with the QBCP at the chemical potential 
confirm the conclusion from the mean-field analysis by 
Sun et al. [11], mainly obtained for the spinless case. 
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They show that the dominant weak-coupUng instabih- 
ties of a inany-fermion system are not only determined 
by the shape of the dispersion, but that also the wave- 
vector-dependent orbital composition of the bands has a 
decisive impact on the preferred ordering tendency. This 
holds as well for linear band crossing points, also known 
as Dirac points. Here, however, the density of states van- 
ishes at the Fermi level, and nonzero, possibly too large 
interactions strengths are required to find instabilities. 

Further research should address where quadratic and 
other band crossing points can be found in realistic band 
structures near the Fermi level in order to investigate 
their potential instabilities. Bilayer graphene is known 
to provide two QBCPs at the K and K' points of the 
Brillouin zone 26], at least if trigonal warping is ig- 
nored. The different spontaneous quantum Hall states 
in A^-layer graphene systems have been classified re- 
cently in Ref. [l^l, irrespective of what interactions 
might be necessary to stabilize these states. Naively, 
we suspect however that the bilayer case of two QBCPs 



with short-range interactions will be dominated by insta- 
bilities with the wave-vector connecting the degeneracy 
points, leading to density wave states, as found in a re- 
lated approach recently by Vafek [22]. Interestingly, for a 
screened Coulomb interaction with dominant scattering 
with small wave- vector transfer within the neighborhood 
of given band crossing point, SN and QAH instabilities 
were also reported for the bilayer situation [22, 23]. Also, 
mean-field-|- fluctuations studies of the bilayer model with 
long-range Coulomb interaction support the possibility of 
QAH and QSH states [11] . Our study with a single band 
crossing connects well to these finding and shows that 
if the two crossing point regions are not at all connected 
by scattering, even QSH instabilities are possible. Future 
work should map out the full phase diagram of the bi- 
layer system depending on the screening and also trigonal 
warping. 

This work was supported by the DFG research unit 
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